Focus sensing apparatus, focus sensing method using phase-differential detection and computer-readable storage medium therefor

ABSTRACT

An apparatus comprising: an image sensing device having a plurality of pixels for receiving light from an object upon separating the light into respective ones of different color components; an optical guidance device for accepting light rays in different directions from the same part of the object and guiding the light rays to the image sensing device; and a phase-difference determining device for determining a phase difference in an output signal of the image sensing device with respect to each of the light rays accepted in the different directions by the optical guidance device, determination being performed based upon a signal obtained by combining output signals of pixels, among the plurality of pixels, that correspond to prescribed different color components.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] This invention relates to a focus sensor using phase-differentialdetection for an imaging device, which uses an image sensing element, ofa digital camera or the like, a method of sensing focus by usingphase-differential detection and a computer-readable storage medium forsuch method.

[0003] 2. Description of Related Art

[0004] A large number of focus sensors that rely upon phase-differentialdetection are available as auto focus devices used in traditionalsingle-lens reflex silver-halide cameras.

[0005]FIG. 19 is a sectional view showing a single-lens reflex camerahaving such a focus sensor using phase-differential detection accordingto the prior art. A ray bundle 109 a emergent from a taking lens 100 issplit into a ray bundle 109 b by reflection at a main mirror 102comprising half-mirror and a ray bundle 109 e obtained by transmissionthrough the main mirror 102. The reflected ray bundle 109 b forms theimage of a subject on the diffusing surface of a focusing plate 103. Thephotographer is capable of observing the image of the subject on thefocusing plate 103 via eyepieces 105 a, 105 b and a pentagonal prism104.

[0006] The ray bundle 109 e obtained by transmission through the mainmirror 102 is reflected by a sub mirror 106 and introduced to a focussensor 107. The latter senses the state of focus (the amount ofdefocusing) of the taking lens 100 with respect to a silver-saltemulsion film 108 by a ray bundle 109 f from the taking lens 100.

[0007] If it is determined that the amount of defocusing sensed isgreater than a predetermined range of focuses and is indicative of adefocused state, a focusing lens of the taking lens 100 is driven so asto eliminate the amount of sensed defocusing, whereby focusing isachieved.

[0008] Focus sensing processing in the conventional focus sensor will bedescribed with reference to FIGS. 20A to 20C and FIGS. 21A to 21C. FIG.20A illustrates the in-focus state. Here ray bundles 116 a, 116 b thathave passed through two different pupils of the taking lens 100 form animage on a primary imaging plane 114, and the subject image on theprimary imaging plane is formed again on a sensor plane, on which twoline sensors 113 a, 113 b are disposed, by secondary image forminglenses 112 a, 112 b, respectively. A field lens 111 is placed in thevicinity of the primary imaging plane 114 of the taking lens 100 andintroduces a pencil of rays of a prescribed image height to the sensorplane in an efficient manner to prevent a decline in quantity of lightcaused by an increase in image height.

[0009] In general, diaphragms (not shown) are placed directly in frontof or directly in back of the secondary image forming lenses 112 a, 112b to limit the two ray bundles 116 a, 116 b that have passed through thedifferent pupils of the taking lens 100. The taking lens 100 does notpossess a member for pupil partitioning.

[0010] Since the two images formed on the line sensors 113 a, 113 b arethe result of ray bundles that have passed through different pupils, therelative positions of the images differ depending upon the amount oflens movement and result in an in-focus state, front-focused state orrear-focused state, as illustrated in FIGS. 20A to 20C and FIGS. 21A to21C.

[0011] In FIGS. 20A and 21A, the spacing between the two images formedon the line sensors 113 a, 113 b in the in-focus state is equal to therelative distance e0 between the two line sensors and is constant at alltimes in the in-focus state.

[0012] In FIGS. 20B and 21B, spacing e1 between the two images is lessthan e0 in the front-focused state where the amount of defocusing is d1.If the defocusing amount d1 is increased, a difference δ1 between e0 ande1 also increases.

[0013] In FIGS. 20C and 21C, spacing e2 between the two images isgreater than e0 in the rear-focused state where the amount of defocusingis d2. If the defocusing amount d2 is increased, a difference δ2 betweene2 and e0 also increases.

[0014] Thus, the amount of defocusing and the direction thereof can bedetermined from the spacing between the two images. The differencebetween the spacing e between the two images in the currently defocusedstate and the reference spacing e0 in the in-focus state, namely theamount of relative shift (phase difference) between the two images givenby δ=e−e0, is calculated by obtaining the correlation between the outputsignals of the two line sensors 113 a, 113 b, the amount of defocusingof the optical system and the direction of this defocusing are foundfrom the phase difference and the focusing lens is controlledaccordingly to achieve the in-focus state.

[0015] When this auto focus scheme is used in an image sensing devicethat employs an image sensing element for photography in a video cameraor digital camera as disclosed in the specification of Japanese PatentApplication Laid-Open No. 9-43507, there is no need to provide alight-receiving sensor for phase-difference detection, as in theabove-mentioned silver-halide camera, and the image sensing element canbe used as the light-receiving sensor for auto3507, there is no need toprovide a light-receiving sensor for phase-difference detection, as inthe above-mentioned silver-halide camera, and the image sensing elementcan be used as the light-receiving sensor for auto focus.

[0016] If the image sens-halide camera, and the image sensing elementcan be used as the light-receiving sensor for auto focus.

[0017] If the image sensing element is for black and white, the outputof the element is the luminance signal representing the image of thereceived light. No problems arise, therefore, since the output obtainedis similar to that of the auto focus sensor for the aforesaidsilver-halide camera. However, if image sensing elements are for color,luminance signals classified according to prescribed color componentsare output from respective ones of the image sensing elements.Consequently, depending upon the color of the subject imaged, it is notpossible to detect the phase difference of the image of the receivedlight.

[0018] Further, if the image sensing element for taking pictures is usedalso as a li if image sensing elements are for color, luminance signalsfor color, luminance signals classified according to prescribed colorcomponents are output from respective ones of the image sensingelements. Consequently, depending upon the color of the subject imaged,it is not possible to detect the phase difference of the image of thereceived light.

[0019] Further, if the image sensing element for taking pictures is usedalso as a light-receiving sensor for detection of phase difference, theoutput of the image sensing element is subjected to prescribed imageprocessing such as a gamma correction to obtain an image signal forphotography. Consequently, if it is attempted to detect phase differenceusing the image signal to which such processing has been applied, autofocus speed slows down because such processing takes time.

SUMMARY OF THE INVENTION

[0020] According to one aspect of the present invention, there isprovided an apparatus comprising: an image sensing device having aplurality of pixels for receiving light from an object upon separatingthe light into respective ones of different color components; an opticalguidance device for accepting light rays in different directions fromthe same part of the object and guiding the light rays to the imagesensing device; and a phase-difference determining device fordetermining a phase difference in an output signal of the image sensingdevice with respect to each of the light rays accepted in the differentdirections by the optical guidance device, determination being performedbased upon a signal obtained by combining output signals of pixels,among the plurality of pixels, that correspond to prescribed differentcolor components; wherein phase difference can be detected properlyirrespective of the color of the object even in cases where an imagesensing element for color is used as a sensor for detecting phasedifference.

[0021] According to another aspect of the present invention, there isprovided an apparatus comprising: an image sensing device for receivinglight from an object; a signal processor for processing an output signalof the image sensing device, which has received the light from theobject, to an image signal for photography; an optical guidance devicefor accepting light rays in different directions from the same part ofthe object and guiding the light rays to the image sensing device; and aphase-difference determining device for determining a phase differencein an output signal of the image sensing device with respect to each ofthe light rays accepted in the different directions by the opticalguidance device, determination being performed based upon an outputsignal of the image sensing device not subjected to the processing bythe signal processor; wherein there is no slow-down in phase-differencedetection speed even in cases where an image sensing device forphotography is used as a sensor for detecting phase difference.

[0022] Other features and advantages of the present invention will beapparent from the following description taken in conjunction with theaccompanying drawings, in which like reference characters designate thesame or similar parts throughout the figures thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

[0023]FIG. 1 is a diagram showing the construction of a focus sensingapparatus according to the present invention as well as the manner inwhich a camera uses this apparatus;

[0024]FIGS. 2A to 2C are diagrams showing the positional relationshipbetween a focus sensing diaphragm and a light shield;

[0025]FIG. 3 is a diagram showing the construction of an interline CCD;

[0026]FIG. 4 is a diagram showing image sensing areas of a CCD;

[0027]FIG. 5 is a timing chart of a vertical synchronizing interval in acase where the vertical charge transfer elements of the CCD are drivenusing four phases;

[0028]FIGS. 6A to 6C are diagrams useful in describing processing when aphase-difference detecting signal is obtained (here the pitch ofphase-difference signals is two pixels of the CCD);

[0029]FIGS. 7A to 7C are diagrams useful in describing processing when aphase-difference detecting signal is obtained (here the pitch ofphase-difference signals is one pixel of the CCD);

[0030]FIGS. 8A to 8C are diagrams useful in describing processing when aphase-difference detecting signal is obtained in a case where thesubject is dark and sensitivity necessary for sensing phase differencecannot be obtained (here the pitch of phase-difference signals is twopixels of the CCD);

[0031]FIGS. 9A to 9C are diagrams useful in describing processing when aphase-difference detecting signal is obtained in a case where thesubject is dark and sensitivity necessary for detecting phase differencecannot be obtained (here the pitch of phase-difference signals is onepixel of the CCD);

[0032]FIGS. 10A to 10C are diagrams useful in describing processing whena phase-difference detecting signal is obtained in a case whereauxiliary light is used;

[0033]FIGS. 11A to 11C are diagrams useful in describing a focus sensingprinciple;

[0034]FIGS. 12A to 12C are diagrams useful in describing a focus sensingprinciple;

[0035]FIGS. 13A to 13C are diagrams useful in describing a focus sensingprinciple;

[0036]FIG. 14 is a diagram useful in describing how to perform acorrelation when a phase difference is obtained by a correlationcalculation;

[0037]FIG. 15 is a waveform diagram showing the relationship between thelateral position of images and a phase difference between images due torelative movement between a camera and a subject;

[0038]FIG. 16 is a characteristic diagram for describing a method ofcorrecting phase-differential detection error;

[0039]FIGS. 17A and 17B are diagrams useful in describing downsamplingof data used in correlation calculation;

[0040]FIG. 18 is a diagram for describing shifting of pixels;

[0041]FIG. 19 is a diagram showing the construction of a single-lensreflex camera having a focus sensing apparatus using aphase-differential detection method according to the prior art;

[0042]FIGS. 20A to 20C are diagrams for describing a focus sensingprinciple according to the prior art; and

[0043]FIGS. 21A to 21C are diagrams for describing the focus sensingprinciple according to the prior art.

DESCRIPTION OF THE PREFERRED EMBODIMENT

[0044]FIG. 1 is a diagram showing the construction of a focus sensingapparatus according to the present invention as well as the manner inwhich a camera uses this apparatus. Shown in FIG. 1 is a focusing lensgroup 1 b of a taking lens, a lens group 1 a other than the focusinglens group 1 b, and a lens feed mechanism 2 for feeding the focusinglens group 1 b. The mechanism 2 includes a motor for moving the lens.Also shown are a focus sensing diaphragm 3 having two apertures 3 a, 3b, a motor 4 for inserting the focus sensing diaphragm 3 in the opticalpath, a light shield 5 for shielding either of the apertures 3 a, 3 b ofthe focus sensing diaphragm 3 from light, a motor 6 for moving the lightshield 5, an optical low-pass filter 7, an infrared blocking filter 8,and a CCD 9 serving as an image sensing element for photoelectricallyconverting an optical image, which has been formed on its imagingsurface, to an electric signal. A color filter shown in FIG. 6A anddescribed later is provided on the imaging surface of the CCD 9.

[0045] An amplifier 10 is for amplifying the output of the CCD 9, and anA/D converter 11 converts, to digital data, the analog signal amplifiedat a prescribed gain by the amplifier 10. A digital signal processor 12subjects the digital signal from the A/D converter 11 to various digitalsignal processing, a system controller 13 performs overall control ofthe camera, and a CCD driver 14 controls driving of the CCD 9 and setsthe amplification factor of the amplifier 10. A lens controller 15controls the movement of the focusing lens group 1 b.

[0046] A buffer memory 16 such as a DRAM is used when temporarilystoring the digital signal, by way of example. Numeral 17 denotes a cardslot and control to which a recording medium and function card, etc.,are connected, and 18 an electronic viewfinder (EVF) whose liquidcrystal display (LCD) is driven a driver 19. A D/A converter 20 is forsending an analog signal to the driver 19. A VRAM 21 retains an imagedisplayed on the electronic viewfinder 18 and outputs a digital signalto the D/A converter 20. An external monochromatic liquid crystal device(LCD) 22 displays camera settings and the like, and an LCD driver 23drives the display of this LCD. An operating switch 24 is for settingthe taking mode of the camera and for sensing a release operation.

[0047] A description primarily of the focus sensing method and focussensing apparatus directly related to the present invention will now berendered with reference to FIG. 1.

[0048] Assume that the power has been introduced to the camera and thatthe camera is capable of photography. In order to perform pupiltime-sharing phase-differential auto focusing, the focus sensingdiaphragm 3 has the two identically shaped apertures side by side in thehorizontal direction. (The aperture on the left side as seen from theside of the CCD 9 shall be referred to the “left pupil 3 a” below, andthe aperture on the right side shall be referred to as the “right pupil3 b” below.) When focus is sensed, the focus sensing diaphragm 3 isinserted into the optical path of the taking lens by the motor 4, andthe light shield 5 is moved by the motor 6 to shield the left pupil 3 aor right pupil 3 b from light so that an optical image comprising abundle of rays that has passed through different pupil areas can beformed on the CCD 9.

[0049] In order for pupil time-sharing phase-differential auto focusingto be carried out, the focus sensing diaphragm 3 is inserted into theoptical path by instructions from the system controller 13. FIGS. 2A to2C are diagrams showing the positional relationship between the focussensing diaphragm 3 and light shield 5. In FIG. 2A, which shows theconditions that prevail at the time of photography, the focus sensingdiaphragm 3 and light shield 5 are situated at positions outside of theoptical path of the taking lens. Numeral 25 indicates the shape of thepupil obtained when the taking diaphragm of the taking lens has beenopened.

[0050] First, as shown in FIG. 2B, the right pupil 3 b of the focussensing diaphragms 3 is blocked by the light shield 5 and the left pupil3 a of the taking lens is opened so that the optical image comprisingthe bundle of rays passing through the left pupil 3 a is formed on theCCD 9, whereby the image is captured. The focus sensing image datacomprising the bundle of rays that has passed through the left pupil 3 ashall be referred to as “left image 1”.

[0051] Next, in order to obtain focus sensing image data comprising abundle of rays that has passed through a different pupil area, the motor6 is actuated to move the light shield 5 in the manner shown in FIG. 2C,thereby opening the right pupil 3 b of the taking optical system so thatthe optical image comprising the bundle of rays passing through theright pupil 3 b is formed on the CCD 9, whereby the image is captured.The focus sensing image data comprising the bundle of rays that haspassed through the right pupil 3 b shall be referred to as “right image2”.

[0052] Further, in order to capture focus sensing image data that haspassed through the left different pupil 3 a and that is identical withthat of the left image 1, the light shield 5 is again moved as shown inFIG. 2B and the image formed on the CCD 9 at this time is captured. Thefocus sensing image data comprising the bundle of rays that has passedthrough the left pupil 3 a at this time shall be referred to as “leftimage 3”.

[0053] A right image 4 and a left image 5 are then captured in similarfashion.

[0054] Exposure when capturing the focus sensing image data is carriedout by providing an electronic shutter, a mechanical shutter (notshown), gain adjustment means for the amplifier 10, summing read-outmeans (not shown) in the CCD 9 and, in certain cases, auxiliary light,and making adjustments using these means.

[0055] Though sensing of focus is performed by capturing the left image1, right image 2, left image 3, right image 4 and left image 5constituting the focus sensing image data, these items of image data arecaptured in a time series. In order to reduce focus sensing error owingto relative movement between the camera and subject (hand movement ormovement of the subject, etc.), the intervals at which these images arecaptured should be made as short as possible. Accordingly, in regard tothe reading out of the focus sensing image data, the read-out will taketoo much time if the entire screen of the CCD 9 is read out, as whenreading out an image for photography. For this reason, only the part ofthe image necessary for sensing focus is read out at a speed higher thanthat of read-out in the case of ordinary photography.

[0056] Such a read-out method will now be described.

[0057]FIG. 3 is a schematic view illustrating an interline-type CCD.Shown in FIG. 3 are a pixel 31, a vertical charge transfer element 32, ahorizontal charge transfer element 33 and as an output section 34.Signal charges obtained by photoelectric conversion at the pixels 31 aresent to the vertical charge transfer elements 32 and the charges aretransferred in the direction of the horizontal charge transfer element33 in order by four-phase driving pulses φV1, φV2, φV3 and φV4. Thehorizontal charge transfer element 33 transfers the signal charge of onehorizontal row, which has been transferred from the vertical chargetransfer elements 32, to the output section 34 by two-phase drivingpulses φH1, φH2, whereby the charge is converted to voltage and output.

[0058]FIG. 4 is a schematic view of image sensing areas of the CCD. Inorder to obtain a high-speed read-out operation, the present embodimentis such that only a readout area necessary for sensing focus is read outat ordinary speed. The charge in other areas is swept out andtransferred at high speed. In FIG. 4, numeral 41 denotes an area readout at ordinary speed for sensing focus, and numerals 42, 43 denotefirst and second halves, respectively, of high-speed sweep-out transferareas.

[0059]FIG. 5 is a timing chart of a vertical synchronizing interval in acase where the vertical charge transfer elements 32 of the CCD aredriven using four phases. Here VD represents a vertical synchronizingsignal, the vertical blanking interval of which is indicated by the lowpotential, and HD represents a horizontal synchronizing signal, thehorizontal blanking interval of which is indicated by low potential.Further, φV1, φV2, φV3 and φV4 represent four-phase driving pulses ofthe focus sensing diaphragm 32, and numerals 51, 52 denote read-outpulses for transferring signal charges obtained by photoelectricconversion at the pixels 31 to the horizontal charge transfer element33. Numerals 53, 54 denote high-speed sweep-out transfer pulsescontained in the four-phase driving pulses. The transfer pulses 53, 54are for high-speed transfer of the signal charges read out to thevertical charge transfer elements 32 from the areas 42 and 43 in FIG. 4.By thus sweeping out at high speed areas other than those requiringreadout, it is possible to speed-up the partial read-out operation.

[0060] When a signal for sensing focus is read out, it is possible toadd the signal charges of a plurality of lines using the horizontalcharge transfer element 33 and read out the sum of the plurality oflines. This is to compensate for an inadequate quantity of light causedat the time of focus detection because the pupils 3 a, 3 b of the focussensing diaphragm 3 are smaller than when the taking diaphragm is openedat the time of photography; it is used to raise the gain of amplifier 10and to improve sensitivity.

[0061] A correlation operation between images is performed using theleft image 1, right image 2, left image 3, right image 4 and left image5 constituting the focus sensing image data thus obtained by high-speedread-out, and the phase differences between the images is obtained. Thefocus is sensed as a result. However, unlike a monochromatic sensorexclusively for sensing focus as used in a single-lens reflexsilver-halide camera, a color filter is incorporated in a single colorsensor, which serves as the taking image sensing element, in a casewhere the taking image sensing element is also used as the focus sensor.In order to use the output signal of the CCD as a phase-differencedetecting signal, therefore, signal processing must be applied.

[0062] As mentioned above, the luminance signal produced using theluminance signal processing circuit for photography can be employed asthe phase-difference detecting signal. However, when the time needed forfocus detection is taken into consideration, it is preferred that thetime required for processing to produce the phase-difference detectingsignal from the output signal of the image sensing element be short.Thus, using the luminance signal processing circuit for photography asthe signal for detecting phase difference involves difficulty.

[0063] Processing for obtaining the phase-difference detecting signalfor sensing focus from the output signal of the CCD will now bedescribed.

[0064]FIG. 6A is a diagram showing a color filter array incorporated inthe CCD. If the four colors Ye, Cy, My, G (yellow, cyan, magenta, green)constituting the color filters are added one pixel at a time, we haveYe+Cy+Mg+G=2B+3G+2R. Accordingly, the average of the sum of the outputsof one block of 2×2 neighboring pixels is used as the signal fordetecting phase difference.

[0065] In actuality, it is possible to perform line summing and read-outwhen the charge from the CCD 9 is read out, as mentioned above. Bysumming and reading out two lines, therefore, two pixels are added inthe vertical direction in the CCD (FIG. 6B), this analog signal isconverted to a digital signal by the A/D converter 11, and two pixelsare added in the horizontal direction in the digital signal processor 12(FIG. 6C), whereby there is obtained a phase-difference detecting signalwhich is the average of the sum of the outputs of one block of 2×2pixels.

[0066] In FIGS. 6A to 6C, the pitch of the phase-difference signals is 2p, where p is the pixel pitch of the CCD. However, digital processingmay be performed so as to make the pitch of the phase-differencedetecting signals p, as shown in FIGS. 7A to 7C.

[0067] In order to improve the S/N ratio in this embodiment, verticalsumming is performed within the CCD. However, read-out may be performedwithout adding lines and vertical summing may be executed by digitalsignal processing.

[0068] Described next will be processing in which, when the subject isdark and the sensitivity necessary for detecting phase difference cannotbe obtained, the number of summed lines at the time of read-out from theCCD 9 is made four lines instead of two lines to obtain thephase-difference detecting signal.

[0069]FIG. 8A is a diagram showing a color filter array incorporated inthe CCD. If the four colors Ye, Cy, My, G constituting the color filtersare added two pixels at a time, we have 2×(Ye+Cy+Mg+G)=2×(2B+3G+2R).Accordingly, the average of the sum of the outputs of one block of 4×2neighboring pixels is used as the signal for detecting phase difference.This enlarges the pixel aperture for obtaining one phase-differencedetecting signal and makes it possible to raise sensitivity as a result.

[0070] In actuality, the number of summed lines at the time of read-outfrom the CCD 9 is changed from two lines to four lines, as mentionedabove, whereby four pixels are added in the vertical direction in theCCD (FIG. 8B), this analog signal is converted to a digital signal bythe A/D converter 11, and two pixels are added in the horizontaldirection in the digital signal processor 12 (FIG. 8C), whereby there isobtained a phase-difference detecting signal which is the average of thesum of the outputs of one block of 4×2 pixels.

[0071] In FIG. 8C, the pitch of the phase-difference signals is 2p,where p is the pixel pitch of the CCD. However, digital processing maybe performed so as to make the pitch of the phase-difference detectingsignals p, as shown in FIG. 9C.

[0072] In order to improve the S/N ratio in this embodiment, verticalsumming is performed within the CCD. However, read-out may be performedwithout adding lines and vertical summing may be executed by digitalsignal processing.

[0073] Described next will be processing in which, when the subject isdark and focus is detected upon introducing auxiliary light for focusdetection, the phase-difference detecting signal is obtained from theoutput signal of the CCD.

[0074] There are cases where the spectrum of the auxiliary light forfocus detection has a spectrum on the side of long wavelengths. In suchcase, rather than using all of the colors Ye, Cy, Mg, G constituting thecolor filters on the CCD 9, it is better to select output signals of thecolor filters for which the transmittance of the spectrum on the longwavelength side is high and eliminate output signals of the colorfilters for which the above-mentioned transmittance is low, therebyimproving focus detection precision. Accordingly, processing forobtaining the phase-difference detecting signal is changed, weightingused when summing output signals from the pixels of each color ischanged to give precedence to the long wavelength side, and calculationis performed using the filter output signals of M and Ye.

[0075]FIG. 10A is a diagram showing a color filter array incorporated inthe CCD. In one block of 2×2 neighboring pixels of the four colors Ye,Cy, Mg, G constituting the color filters, adding the colors upon making0 the weighting of Cy, G and 1 the weighting of Mg, Ye gives us(1×Ye+0×Cy+1×Mg+0×G). Accordingly, the average of the sum of the outputsof one pixel each of My, Ye for which the transmittance of the spectrumon the long wavelength side is high is used as the phase-differencedetecting signal.

[0076] In actuality, rather than performing line summing and read-out(FIG. 10B) when charge is read out of the CCD, this analog signal isconverted to a digital signal by the A/D converter 11, the weighting ofCy, G is made 0 and the weighting of My, Ye is made 1 within the digitalsignal processor 12, Mg, Ye are extracted and processing is executed(FIG. 10C), thereby obtaining a phase-difference detecting signal whichis the average of the sum of the outputs of Mg, Ye in one block of

[0077] 2×2 pixels.

[0078] Next, using the phase-difference detecting signal obtained in themanner described above, a correlation operation is performed to find thephase difference.

[0079]FIGS. 11A to 11C, 12A to 12C and 13A to 13C are diagrams useful indescribing the focus sensing principle of pupil time-sharingphase-differential AF according to this embodiment. FIGS. 11A to 11Cillustrate the in-focus state. In the picture-taking state shown in FIG.1C, a ray bundle 27 that has passed through a taking lens 1 has itsfocal point formed on the light-receiving surface of the CCD 9 and theamount of defocusing is zero.

[0080] In FIG. 11A, the focus sensing diaphragm 3 is inserted into theoptical path of the taking lens, the right pupil 3 b is shielded fromlight by the light shield 5, the left pupil 3 a is opened and a raybundle 27 a that has passed through the left pupil 3 a forms an image onthe light-receiving surface of the CCD 9 at a position having a distanceof zero from the optical axis 26. In FIG. 11B, the left pupil 3 a isshielded from light by the light shield 5, the right pupil 3 b is openedand a ray bundle 27 b that has passed through the right pupil 3 b formsan image on the light-receiving surface of the CCD 9 at a positionhaving a distance of zero from the optical axis 26. Thus, in thein-focus state, optical images comprising ray bundles that have passedthrough the different pupils 3 a, 3 b both impinge upon thelight-receiving surface of the CCD 9 at the same position having zerodistance from the optical axis 26. Accordingly, the phase differencebetween these two images is zero.

[0081]FIGS. 12A to 12C illustrate the front-focused state. In thepicture-taking state shown in FIG. 12C, the ray bundle 27 that haspassed through a taking lens 1 has its focal point formed a distance din front of the light-receiving surface of the CCD 9. In FIG. 12A, thefocus sensing diaphragm 3 is inserted into the optical path of thetaking lens, the left pupil 3 a is opened and the ray bundle 27 a thathas passed through the left pupil 3 a forms an image on thelight-receiving surface of the CCD9 at a position having a distance of+δ/2 from the optical axis 26. In FIG. 12B, the right pupil 3 b isopened and the ray bundle 27 b that has passed through the right pupil 3b forms an image on the light-receiving surface of the CCD 9 at aposition having a distance of −δ/2 from the optical axis 26. Thus, inthe front-focused state, the phase difference between the two imagescomprising the ray bundles that have passed through the two differencepupils 3 a, 3 b is (+δ/2)−(−δ/2)=δ.

[0082]FIGS. 13A to 13C illustrate the rear-focused state. In thepicture-taking state shown in FIG. 13C, the ray bundle 27 that haspassed through a taking lens 1 has its focal point formed a distance dto the rear of the light-receiving surface of the CCD 9. In FIG. 13A,the focus sensing diaphragm 3 is inserted into the optical path of thetaking lens, the left pupil 3 a is opened and the ray bundle 27 a thathas passed through the left pupil 3 a forms an image on thelight-receiving surface of the CCD 9 at a position having a distance of−δ/2 from the optical axis 26. In FIG. 13B, the right pupil 3 b isopened and the ray bundle 27 b that has passed through the right pupil 3b forms an image on the light-receiving surface of the CCD 9 at aposition having a distance of +δ/2 from the optical axis 26. Thus, inthe rear-focused state, the phase difference between the two imagescomprising the ray bundles that have passed through the two differencepupils 3 a, 3 b is (−δ/2)−(+δ/2)=−δ.

[0083] Thus, the in-focus state can be sensed by obtaining the phasedifference between two phase-difference detecting signals comprising theray bundles that have passed through the left pupil 3 a and right pupil3 b.

[0084] The phase-difference calculation algorithm will now be described.

[0085] First, in order to simplify the discussion, a case in which therelative positions of the subject and taking optical system are notmoved will be described as an example.

[0086] The phase difference between the phase-difference detectingsignal (left image) comprising the ray bundle that has passed throughthe left pupil area and the phase-difference detecting signal (rightimage) comprising the ray bundle that has passed through the right pupilarea is obtained by a correlation operation. As shown in FIG. 14, thephase-difference detecting signals each comprise M×N signalshorizontally and vertically, respectively, from which m×n signals arecut to perform a correlation operation.

[0087] The correlation is performed as follows: As indicated by Equation(1), the integrated value C(τx, τy) (where τx, τy are positivevariables) of the product of a and b is calculated while holding thecutting position of data a of the left image and successively shiftingthe phase (τx, τy) of data b of the right image with respect to the leftimage, and the value C (τx, τy) obtained every phase (τx, τy) is adoptedas the amount of correlation between the two images. $\begin{matrix}{{C\left( {{\tau \quad x},{\tau \quad y}} \right)} = {\sum\limits_{j = 1}^{n}\quad {\sum\limits_{i = 1}^{m}\quad \left\lbrack {{a\left( {i,j} \right)} \times {b\left( {{i + {\tau \quad x}},{j + {\tau \quad y}}} \right)}} \right\rbrack}}} & (1)\end{matrix}$

[0088] where τx=−Tx . . . 2, −1, 0, 1, 2, . . . , Tx

[0089] τy=−Ty . . . 2, −1, 0, 1, 2, . . . , Ty

[0090] The correlation quantity C(τx, τy) takes on a minimum value andthe phase (τx, τy) at such time corresponds to the phase difference (δx,δy) between the data a of the left image and the data b of the rightimage. Furthermore, in order to obtain the phase (τx, τy), which raisesthe phase-difference detection precision, up to a value below thedecimal point and not the value of a whole number (the case where thephase-difference detecting signal was found so as to make the pitch thesame as that of the CCD pixel pitch, as in FIG. 7), the phase differenceshould be calculated by interpolation using the minimum value of thecorrelation quantity and values on either side thereof.

[0091] In a case where relative positions of the subject and takingoptical system move, δy becomes zero in principle if the effects ofnoise and the like are nil.

[0092] In order to reduce the effects of the pattern and contrast of thesubject, the effects of a difference in photographic conditions, theeffects of the color filter array incorporated in the solid-state imagesensing element and the effects of noise, the correlation operation maybe performed after the phase-difference detecting signal is subjected tofilter processing.

[0093] Since the relationship between phase difference and the amount ofmovement of the image plane and amount of defocusing is decided by theoptical system, focusing is achieved by finding the amount of defocusingfrom the phase difference, finding the amount of lens feed necessary toattain the in-focus state and then controlling the focusing of theoptical system accordingly.

[0094] Described above is a method of sensing focus using two imagescomprises ray bundles that have passed through different pupil areas ina case where there is no relative movement between the subject andoptical system.

[0095] Next, an algorithm for calculating phase difference will bedescribed in a case where there is relative movement between the subjectand optical system. Though the phase difference can be obtained from twoimages (left and right images) comprising bundles of light fromdifference pupil areas, in this embodiment the left and right images arecaptured in a time series and, if there is relative movement between thesubject and the optical system (hand movement or movement of thesubject) and the phase difference will include a focus sensing errorowing to the effects of such movement.

[0096] Accordingly, in regard to the reading out of the focus sensingimage data, it has been contemplated not to read out the entire screenof the CCD, as when reading out an image for photography, but to readout only the part of the image necessary for sensing focus at a speedhigher than that of read-out in the case of ordinary photography, asdescribed earlier. This makes it possible to shorten the image captureinterval as much as possible. However, capturing the signals of the leftand right images involves an operation that differs with time; if thecamera and subject move relative to each other during this operation,the occurrence of an error in the detection of phase difference cannotbe avoided.

[0097] In this embodiment, therefore, five images, namely the left image1, right image 2, left image 3, right image 4 and left image 5, arecaptured as the phase-difference detecting signals in a time series, anda correction of the phase-difference error due to relative movementbetween the camera and subject (hand movement and movement of thesubject) is performed using the plurality of phase differences obtainedfrom neighboring images.

[0098] If the camera and subject move relative to each other, theinfluence thereof appears in the phase difference (δx, δy) between thetwo images. Here δy is the phase difference of the images in thevertical direction and has a direction orthogonal to the direction of aphase shift produced in dependence upon the amount of opticaldefocusing. It therefore does not include a phase-shift component due todefocusing but only phase-shift component due to relative movementbetween the camera and subject. Accordingly, the value of δy is used asis as a quantity for correcting image movement in the vertical directioncaused by relative movement between the camera and subject.

[0099] The phase difference of the images in the horizontal direction isrepresented by δx. This corresponds to the direction of the phase shiftproduced in dependence upon the amount of optical defocusing.Consequently, the value δx obtained from two images that includerelative movement between the camera and subject includes a phase-shiftcomponent due to defocusing and a phase-shift component due to relativemovement between the camera and subject.

[0100]FIG. 16 illustrates the relationship between position of an imagein the horizontal direction (x direction) due to relative movementbetween the camera and subject and phase difference between the imagesafter image movement due to relative movement between the camera andsubject in the vertical direction has been corrected. The solid linesL1, R2, L3, R4, L5 are the positions of the left image 1, right image 2,left image 3, right image 4 and left image 5, respectively, captured byswitching between pupils in a time series in order to detect the phasedifference. Further, t=t1, t2, t3, t4, t5 denote the capture times ofthe respective images; the intervals at which the images are capturedare equal. The broken lines R1′, L2′, R3′, L4′, R5′ indicate thepositions of images comprising ray bundles that have passed throughdifferent pupils on the assumption that capture is performed at the sametime. In actuality, these are signals for which capture is not possible.

[0101] Further, m1, m2, m3, m4 denote amounts of image movement due torelative movement between the camera and subject during the time ofimage capture of each image, and δ represents the phase-shift componentdue to defocusing of the optical system; it is the phase differencesought for sensing the focus.

[0102] First, phase differences δ1, δ2, δ3, δ4 between the images L1 andR2, R2 and L3, L3 and R4, R4 and L5, respectively, comprising raybundles from different pupils are obtained using the left image as areference. As will be understood from FIG. 15, each phase differenceobtained includes a phase-shift component due to defocusing and aphase-shift component due to relative movement between the camera andsubject.

[0103]FIG. 16 is a diagram for describing a method of correctingphase-difference detection error. Here time t is plotted along thehorizontal axis and position x of the image is plotted along thevertical axis. Further, L1, R2, L3, R4, L5 represent positions of thesubject image captured in a time series. By applying aquadratic-function approximation to movement of the subject image due torelative movement between the camera and subject during signal capture,the positions L2′, L4′ of the subject of the left image at times t2, t4,respectively, are obtained. The average value of the phase differenceδ2′ between L2 and R2 and the phase difference δ4′ between L4 and R4′ isfinally obtained and is adopted as the phase difference δ.

[0104] If the quadratic function y=At²+Bt passing through (t1, x1), (t3,x3), (t5, x5) is obtained as t1=0, x1=0 and the finally obtained phasedifference δ found from the average value of δ2′ and δ4′, the captureintervals of the images will be equal. The phase difference δ aftercorrection, therefore, is as indicated by Equation (2) below.

δ=(δ1+3×δ2+3×δ3+δ4)/8  (2)

[0105] The amount of lens feed necessary to achieve the in-focus stateis obtained from the thus found phase difference δ corrected forrelative movement between the camera and subject, the optical system issubjected to focusing control accordingly, and photography is carriedout after the in-focus state is attained.

[0106] In performing photography, the picture is taken after the focussensing diaphragm 3 is withdrawn from the optical path of the takinglens. In order to record the image, the data is read out of the CCD,subjected to image signal processing by the digital signal processor 12and, if necessary, subjected to processing such as image datacompression. The resulting image is recorded on a recording medium viathe card slot 13.

[0107] The image data at this time is subjected to video processing bythe digital signal processor 12 in order to be displayed in theviewfinder. The processed data is displayed in viewfinder 18 via theVRAM 21. The photographer can thus check the image of the subject.

[0108] In this embodiment, a phase-difference detecting signal isobtained from one block which includes one pixel of each of the colorsconstituting the color filter array of an image sensing element. As aresult, it is possible to execute processing more simply in comparisonwith complicated matrix processing such as luminance signal processingfor photography. It is possible to speed up processing of thephase-difference detecting signal and reduce the scale of circuitry.

[0109] Further, the average of the sum of one block which includes onepixel of each of the colors constituting the color filter array of animage sensing element is adopted as the phase-difference detectingsignal. This makes simpler processing possible.

[0110] Further, by obtaining a phase-difference detecting signal fromone block composed of mutually adjacent pixels which include one pixelof each color constituting the color filter array of the image sensingelement, simpler processing becomes possible.

[0111] Further, when the phase-difference detecting signal is obtained,weighting used when summing the pixels of each color is changed independence upon the color temperature of the light source, i.e., thesubject. As a result, rather than using output signals of all colorsconstituting the color filters upon weighting these signals identicallyand then averaging them, output signals of color filters for which thetransmittance is high with respect to various conditions such as thecolor temperature of the subject due to auxiliary light are selectedover output signals of color filters for which the transmittance is low,thereby improving focus sensing precision.

[0112] Further, by providing means for moving the pupil area, the imagesensing element for photography can also be used as a sensor for sensingfocus, thus making it possible to conserve space, reduce cost and raisefocus sensing accuracy.

[0113] [Downsampling Processing]

[0114] When a correlation operation for obtaining phase difference isperformed, the data of all phase-difference detecting signals within anarea (m×n areas in FIG. 14) employed in the correlation operation isused as is, as mentioned earlier. The signal pitch of thephase-difference detecting signals at this time is the same as the pixelpitch of the CCD, also as mentioned above. However, in a case where thetaking lens is a zoom lens, the focal length on the telephoto enddiffers from that on the wide-angle end. When these focal lengths arecompared for the same subject distance, it is found that the focallength on the wide-angle end has a greater depth of field and that it isacceptable for the focus sensing precision necessary for sensing focusto be lower on the wide-angle end than on the telephoto end.

[0115] A first example of downsampling processing according to thisembodiment is as follows: When the focus sensing precision on thewide-angle end is one-half the focus sensing precision on the telephotoend, it will suffice to find the phase difference with one-half theprecision. If photography is performed on the wide-angle end, therefore,then, in regard to data in the horizontal direction corresponding to thedirection of the phase shift produced in dependence upon the amount ofdefocusing of the optical system, the data of the m×n areas used in thecorrelation operation is downsampled every other pixel, as shown in FIG.17A, the signal pitch of the phase-difference detecting signals used inthe correlation operation is made one-half the pixel pitch of the CCDand then the correlation operation is carried out. The shaded portionsshown in FIG. 17B represent the data used in the correlation operation.

[0116] As a result of the foregoing, the number of items of data used incorrelation becomes m/2×n, thereby shortening the time needed for thecorrelation operation. Thus, it will suffice to select, in dependenceupon the focal length at which photography is actually performed,whether or not the downsampling of data used in the correlationoperation is carried out.

[0117] This embodiment is described in regard to a zoom lens. However,in case of a camera in which taking lenses can be interchanged, thefocus sensing precision can be changed by changing whether or notdownsampling is performed in dependence upon the focal length of thelens actual used in photography. Though the amount of downsampling is50% in this embodiment, the amount of downsampling may be changed independence upon the focal length of the taking lens.

[0118] In regard to data in the vertical direction of thephase-difference detecting signal, downsampling may be carried in thevertical direction in a case where it is permissible for the correctionprecision of the image in the vertical direction due to relativemovement between the camera and subject to be low.

[0119] The amount of lens feed necessary to achieve the in-focus stateis obtained from the thus found phase difference δ corrected forrelative movement between the camera and subject, the optical system issubjected to focusing control accordingly, and photography is carriedout after the in-focus state is attained. In performing photography, thepicture is taken after the focus sensing diaphragm 3 is withdrawn fromthe optical path of the taking lens. In order to record the image, thedata is read out of the CCD 9, subjected to image signal processing bythe digital signal processor 12 and, if necessary, subjected toprocessing such as image data compression. The resulting image isrecorded on a recording medium via the card slot 13.

[0120] The image data at this time is subjected to video processing bythe digital signal processor 12 in order to be displayed in theviewfinder. The processed data is displayed in viewfinder 18 via theVRAM 21. The photographer can thus check the image of the subject.

[0121] A second example of downsampling processing according to thisembodiment will now be described.

[0122] The second example of downsampling processing involves selecting,in dependence upon the F-number of the taking lens, whether or not toperform downsampling of the data of the phase-difference detectingsignal used in the correlation operation for obtaining the phasedifference.

[0123] Only the differences between this processing as that of the firstexample of downsampling will be described.

[0124] When the aperture of the taking lens is stopped down, depth offield increases. It will be understood, therefore, the focus sensingprecision necessary for sensing focus differs depending upon theF-number of the taking lens when a picture is taken. In general, theallowed amount of defocusing is F×R, where F represents the F-number ofthe taking lens and R the radius of the least circle of confusion. Thelarger the F-number F (the more the aperture is stopped down), thegreater the allowable amount of defocusing, the lower the focus sensingprecision needed and, hence, the lower the phase-difference detectionprecision needed.

[0125] Accordingly, when the phase difference is obtained, the method ofdownsampling the data of the phase-difference detecting signals in thearea used in the correlation operation is changed in dependence upon theF-number at the time of photography, thereby shortening the time neededfor the correlation operation.

[0126] For example, if a picture is taken in a case where the openaperture F-number of the taking lens is F2, the data employed in thecorrelation operation uses the phase-difference detecting signals thatprevailed prior to downsampling, as shown in FIG. 17A. Since the signalpitch of the phase-difference detecting signals is the same as the pixelpitch of the CCD, the pitch of the data used in the correlationoperation is equal to the pixel pitch of the CCD and the number of itemsof data is m×n.

[0127] Next, if a picture is taken in a case where the F-number of thetaking lens is F4, the operation is performed after the data used in thecorrelation operation is downsampled every other pixel, a shown in FIG.17B. The shaded portions represent the downsampled data used in thecorrelation operation. The pitch of the data used in the correlationoperation at this time is twice the CCD pixel pitch and the number ofitems of data is m/2×n. In other words, the data is reduced by one-half.

[0128] By thus downsampling the data used in the correlation operationin dependence upon the F-number at the time of photography, processingtime can be shortened.

[0129] A third example of downsampling processing according to thisembodiment will now be described.

[0130] The third example of downsampling processing involves selecting,on the basis of whether photography is performed with the opticallow-pass filter 7 inserted or not at the time of photography, whether ornot to perform interpolation of the data of the phase-differencedetecting signals used in the correlation operation for obtaining thephase difference.

[0131] Only the differences between this processing as that of the firstexample of downsampling will be described.

[0132] The CCD 9 in FIG. 1 is capable of being moved in a planeorthogonal to the optical axis by a pixel shifting mechanism (notshown). Color filters of the kind shown in FIG. 18 are incorporated inthe CCD 9. Assume that the CCD is composed of square pixels having apixel pitch p. The pixel shifting mechanism is capable of moving the CCDin the horizontal and vertical directions of the image by apiezoelectric element. As shown in FIG. 18, an image 1 is captured andthen shifted in the horizontal direction by p/2, an image 2 is capturedand then shifted in the vertical direction by p/2, an image 3 iscaptured and then shifted in the horizontal direction (the directionopposite that of the shift made in order to capture the image 2) by p/2,and an image 4 is captured.

[0133] From these four images it is possible to obtain image data thatcorresponds to raw image data captured at a sampling interval of p/2(where the pixel aperture is equal to the CCD that performedphotography). As a result, a composite image having a resolution higherthan that of one image captured by the CCD can be obtained.

[0134] The optical low-pass filter 7 has a spatial frequencycharacteristic in which the response becomes zero at the Nyquistfrequency, which is one-half the sampling frequency with respect to thepixel pitch p of the CCD 9. The filter 7 functions to suppress aliasingcomponents due to the spatial sampling of the CCD. However, in a casewhere a composite image is obtained by capturing a plurality of imagesby shifting pixels, as described above, photography is performed throughan optical low-pass filter in which the response becomes zero at theNyquist frequency with respect to the pixel pitch p of the CCD.Consequently, the composite image obtained by combining the plurality ofimages is not rendered as a picture having a very high resolution.

[0135] Accordingly, when pixel-shifting photography is performed, thepicture is taken after withdrawing the optical low-pass filter from theoptical path of the taking lens. In other words, the resolving of theimage formed on the CCD in the ordinary photography mode in which thereis no pixel shift differs from that in the photography mode in whichpixels are shifted, and this is accompanied by different focus sensingprecisions as well.

[0136] Thus, in the case of the ordinary photography mode in which apicture is taken with the optical low-pass filter inserted into theoptical path of the taking lens, a low focus sensing precision willsuffice. As shown in FIG. 17B, therefore, calculation is performed afterthe data used in the correlation calculation is downsampled every otherpixel. The shaded portions represent the downsampled data used in thecorrelation operation. The pitch of the data used in the correlationoperation at this time is twice the CCD pixel pitch and the number ofitems of data is m/2×n. In other words, the data is reduced by one-half.

[0137] In the case of the pixel shifting photography mode in which apicture is taken with the optical low-pass filter withdrawn from theoptical path of the taking lens, it is necessary to raise the focussensing precision. As shown in FIG. 17A, therefore, the data used in thecorrelation operation employs the phase-difference detecting signalsprevailing prior to downsampling. The pitch of the data used in thecorrelation operation at this time is equal to the pixel pitch of theCCD and the number of items of data is m×n. Thus, when a picture istaken with the optical low-pass filter inserted, the data used in thecorrelation operation is downsampled, thereby making it possible toshorten processing time.

[0138] Thus, in accordance with the downsampling processing of thisembodiment as described above, downsampling means for downsampling thedata of the phase-difference detecting signals in an area used in thecorrelation operation is provided when performing a correlationoperation for obtaining the phase difference in order to sense focus andwhen performing a correlation operation to obtain the data ofphase-difference detecting signals in an area used in the correlationoperation. As a result, whether or not data used in the correlationoperation is to be downsampled can be selected in dependence upon suchphotographic conditions as focal length of the taking lens, F-number atthe time of photography and the degree to which the image can beresolved on the image sensing medium. This makes it possible to detectthe phase difference with a focal-length sensing precision suited to thephotographic conditions, to shorten the time necessary for thecorrelation operation and to shorten the time needed for overall sensingof focus.

[0139] [Correlation Method]

[0140] In the case of a camera system in which the amount of defocusingfalls within the prescribed allowable limits by a single focus detectionoperation and control of the focus of the optical system, photographymay begin without performing a second detection of focus forconfirmation purposes. Since the in-focus state usually is confirmed,however, ordinarily focus is sensed from the second time onward. In suchcase the correlation operation is performed to again obtain the phasedifference. However, performing the same correlation operation thesecond time in a case where the amount of defocusing is smaller afterthe first time represents needless calculation.

[0141] Accordingly, in a first example of a correlation method accordingto this embodiment, the range of a search of image signals at the timeof phase-difference detection is narrowed, from the second focusdetecting operation onward, based upon the amount of defocusing servingas the preceding focus detection information, and the correlationoperation is carried out following the narrowing of the search range. Inactuality, the value of Tx, which is the search range of τx of thecorrelation calculation equation (1), need only be reduced so as tobecome a value obtained by the time of the phase difference δ1 detectedthe first time. When large defocusing is performed in this manner, thesearch range is broadened and then narrowed when a point near thein-focus state is achieved. This makes it possible to shorten the timeneeded for the correlation calculations from the second time onward.

[0142] Here an example has been described in which the (i+1) th searchrange is narrowed to become a value that is found by the time of thephase difference δ1 detected the first time. However, since the phasedifference when the (i+1)th focus detection operation is performedbecomes smaller than the phase difference the first time, it willsuffice to change to a search range obtained by the time of a prescribedphase difference δ1′ (<δ1), with respect to the first phase differenceδ1.

[0143] Thus, after the in-focus state is achieved, the focal-pointsensing diaphragm 3 is withdrawn from the optical path of the takinglens and the picture is taken. In order to record the image, the data isread out of the CCD, subjected to image signal processing by the digitalsignal processor 12 and, if necessary, subjected to processing such asimage data compression. The resulting image is recorded on a recordingmedium via the card slot 13.

[0144] This image data is subjected to video processing by the digitalsignal processor 12 in order to be displayed in the viewfinder. Theprocessed data is displayed in viewfinder 18 via the VRAM 21. Thephotographer can thus check the image of the subject.

[0145] A second correlation method according to this embodiment will nowbe described.

[0146] According to the second correlation method, the number of stepsthe image signal in the correlation operation for obtaining the phasedifference is shifted at the succeeding focus sensing operation is madedifferent from that of the preceding focus sensing operation.

[0147] Only the differences between this method and the first example ofthe correlation method will be described.

[0148] When a state of large defocusing prevails in a case where it isnecessary to perform focus detection and focusing control of the opticalsystem two or more times to attain the in-focus state, first focus issensed in the state of large defocusing and focus is finally sensed in astate near the in-focus state. Accordingly, it will suffice to initiallyperform coarse sensing of focus, move the lens to the vicinity of focusand then perform precise focusing.

[0149] Accordingly, in the first focus sensing operation, the τxshifting method is made τx=−Tx, −Tx+2 . . . , −4, −2, 0, 2, 4, . . . ,Tx−2, Tx in the correlation calculation equation (1) for obtaining thephase difference, and the step quantity for shifting the image signal isdoubled. When the preceding phase difference δ becomes smaller than apredetermined value, the number of steps for shifting the image signalis decremented and the correlation operation is performed. In otherwords, the focus sensing precision is changed in such a manner that atfirst the focus sensing precision is lowered, coarse focusing isperformed and then, when a point near the in-focus state is attained,sensing is performed at a higher precision. Thus, the amount ofcalculation is reduced to the extent that the step quantity iscoarsened, thereby shortening the time needed for the correlationoperation. A similar operation may be carried out in regard to τy aswell.

[0150] A third correlation method according to this embodiment will nowbe described.

[0151] According to the third correlation method, the size of the areaof the phase-difference detecting signals used in the correlationoperation at the succeeding focus sensing operation is made differentfrom that of the preceding focus sensing operation.

[0152] The size of the area of the phase-difference detecting signalsused in the correlation operation usually is fixed at a predeterminedsize. However, in a case where the contrast of the subject is low andphase difference cannot be detected, or in a case where the detectionprecision is poor, enlarging the area causes the edge of the subject tofall within the area, thereby raising the detection precision. However,if the area is too large, a subject having a long subject distance and asubject having a short subject distance tend to occur in the area anddetection precision declines (perspective conflict). The area usually isset to a predetermined size for this reason.

[0153] Accordingly, the result of focus detection up to the precedingtime is checked. If focus detection cannot be carried out or if thefocus state cannot be attained even if detection is performed aprescribed number of times, the size of the area is enlarged by apredetermined value. This prevents wasteful repeated sensing of focus ina state in which the edge of the subject is not within the area andmakes it possible to reduce the number of times focus is sensed. Theresult is shorter calculation time.

[0154] The enlarging of the area may be performed not only in thehorizontal direction but in the vertical direction as well.

[0155] A fourth correlation method according to this embodiment will nowbe described.

[0156] According to the fourth correlation method, whether or not ashift operation in a direction substantially orthogonal to the directionof the phase shift produced in dependence upon the defocusing quantityof the optical system is selected in the correlation operation.

[0157] Only the differences between this method and the first example ofthe correlation method will be described.

[0158] When optical images comprising bundles of rays from differentpupil areas are captured, the time at which the left image is captureddiffers from that at which the right image is captured. For this reason,a case in which there is relative movement between the subject andoptical system is dealt with in the manner described above. However, ina case where there is a large amount of hand movement, the phasedifferences δ1, δ2, δ3, δ4, δ5 develop a variance and the precision withwhich the phase difference δ is detected declines if a shift calculationis performed by the correlation operation in the vertical direction,which is substantially orthogonal to the direction of the phase shiftproduced in dependence upon the amount of defocusing of the opticalsystem.

[0159] In a case where the S/N ratio of the phase-difference detectingsignal is poor, e.g., in a case where the subject is dark, the amount oflight is inadequate and the output of the CCD 9 is enlarged by theamplifier 10, the phase differences δ1, δ2, δ3, δ4, δ5 develop avariance and the precision with which the phase difference δ is detecteddeclines if a shift calculation is performed by the correlationoperation in the vertical direction, which is substantially orthogonalto the direction of the phase shift produced in dependence upon theamount of defocusing of the optical system.

[0160] When the correlation operation is performed under suchphotographic conditions, therefore, a shift calculation in the verticaldirection, which is substantially orthogonal to the direction of thephase shift produced in dependence upon the amount of defocusing of theoptical system, is not performed.

[0161] In actuality, the fact that the S/N ratio is poor is judged fromthe exposure decision value that is for adjusting the amount of light,and the shift calculation in the vertical direction is not carried outwhen such a judgment is made. Further, when the variance in the verticalphase differences δy1, δy2, δy3, δy4, δy5 between the images L1 and R2,R2 and L3, L3 and R4, R4 and L5, respectively, comprising bundles oflight from different pupils exceeds a predetermined value, it is judgedthat hand movement is large and the vertical shift operation is notcarried out.

[0162] Thus, when the precision with which the phase difference δ isdetected will decline if a shift calculation is performed in thevertical direction, which is substantially orthogonal to the directionof the phase shift produced in dependence upon the amount of defocusingof the optical system, the correlation operation in this direction isnot carried out. This makes it possible to prevent a decline indetection precision and shorten calculation time.

[0163] A fifth correlation method according to this embodiment will nowbe described.

[0164] The fifth correlation method is such that in the correlationoperation for obtaining the phase difference, downsampling means isprovided for performing the correlation upon downsampling the data ofthe phase-difference detecting signals in the area used in thecorrelation operation, and whether or not downsampling is to beperformed is selected based upon the result of the preceding focussensing operation.

[0165] Only the differences between this method and the first example ofthe correlation method will be described.

[0166] When a correlation operation for obtaining phase difference isperformed, the data of all phase-difference detecting signals within anarea (m×n areas in FIG. 14) employed in the correlation operation isused as is, as mentioned earlier. The signal pitch of thephase-difference detecting signals at this time is the same as the pixelpitch of the CCD, also as mentioned above.

[0167] However, when a state of large defocusing prevails in a casewhere it is necessary to perform focus detection and focusing control ofthe optical system two or more times to attain the in-focus state, firstfocus is sensed in the state of large defocusing and focus is finallysensed in a state near the in-focus state. In other words, it willsuffice to move the lens coarsely to the vicinity of focus and thenperform precise focusing.

[0168] Accordingly, in regard to data in the horizontal directioncorresponding to the direction of the phase shift produced in dependenceupon the amount of defocusing of the optical system, the first time thefocus sensing operation is performed the data of the m×n areas used inthe correlation operation is downsampled every other pixel, as shown inFIG. 17A, the signal pitch of the phase-difference detecting signalsused in the correlation operation is made one-half the pixel pitch ofthe CCD and then the correlation operation is carried out. The shadedportions shown in FIG. 17B represent the data used in the correlationoperation. When the phase difference δ that is the result of thepreceding focus sensing operation becomes smaller than a predeterminedvalue, the correlation operation is performed without executingdownsampling.

[0169] More specifically, the focus sensing precision is changed in sucha manner that at first the focus sensing precision is lowered, coarsefocusing is performed and then, when a point near the in-focus state isattained, sensing is performed at a higher precision. As a result, thenumber of items of data used in correlation becomes m/2×n, therebyshortening the time needed for the correlation operation.

[0170] Though the amount of downsampling is 50% in this embodiment, theamount of downsampling may be changed depending upon the results offocusing. Further, downsampling may be performed in similar fashion inthe vertical direction as well.

[0171] A sixth correlation method according to this embodiment will nowbe described.

[0172] This method differs from the fifth correlation method in thatwhether or not downsampling is performed is selected based upon thenumber of detection operations and not the results of focus detection upto the preceding time.

[0173] When a correlation operation for obtaining phase difference isperformed, the data of all phase-difference detecting signals within anarea (m×n areas in FIG. 14) employed in the correlation operation isused as is, as mentioned earlier in the description of the firstcorrelation method. The signal pitch of the phase-difference detectingsignals at this time is the same as the pixel pitch of the CCD, also asmentioned above.

[0174] However, when a state of large defocusing prevails in a casewhere it is necessary to perform focus detection and focusing control ofthe optical system two or more times to attain the in-focus state, firstfocus is sensed in the state of large defocusing and focus is finallysensed in a state near the in-focus state. In other words, it willsuffice to move the lens coarsely to the vicinity of focus and thenperform precise focusing.

[0175] Accordingly, in regard to data in the horizontal directioncorresponding to the direction of the phase shift produced in dependenceupon the amount of defocusing of the optical system, as shown in FIGS.17A, 17B, the first time the focus sensing operation is performed thedata of the m×n areas used in the correlation operation is downsampledevery other pixel, as shown in FIG. 17A, the signal pitch of thephase-difference detecting signals used in the correlation operation ismade one-half the pixel pitch of the CCD and then the correlationoperation is carried out. The shaded portions shown in FIG. 17Brepresent the data used in the correlation operation. In focus sensingoperations from the second onward, the correlation operation isperformed without executing downsampling.

[0176] More specifically, when the first focus sensing operation isperformed, at which time the amount of defocusing may be large, thefocus sensing precision is lowered and then focus detection from thesecond time onward is performed with a high precision. By performingdownsampling in this manner, the number of items of data used incorrelation becomes m/2×n in the first focus sensing operation and lesstime is needed for the correlation operation.

[0177] In accordance with the correlation methods of this embodiment, asdescribed above, a plurality of calculation methods for sensing focusare selected and the calculation is performed a plurality of times,thereby making it possible to select a calculation method suited tofocus detection at the time. In comparison with a situation in which onecalculation method is used in all cases, needless calculation can beeliminated. This makes it possible to speed up calculations.

[0178] Further, in accordance with this embodiment, the calculationmethod for the next detection of focus is selected based uponinformation representing the preceding detection of focus. As a result,a succeeding calculation method can be made different from a precedingcalculation method based upon previous focus detection information, thenumber of times focus is detected and the detection conditions. Thismakes it possible to eliminate needless calculation the next time focusis detected and, hence, to shorten calculation time.

[0179] Further, in accordance with the present invention, the searchrange of image signals at the time of phase-difference detection thenext time is made different from the last time in the correlationoperation for obtaining the phase difference. When the amount ofdefocusing is large, the search range at the time of phase-differencedetection can be enlarged; when the amount of defocusing is small, thesearch range at the time of phase-difference detection can be reduced.By setting the next search range based upon the amount of defocusingpreviously, needless calculation can be eliminated and calculation canbe speeded up.

[0180] Further, in accordance with this embodiment, the number of stepsthe image signal at the time of phase-difference detection is shiftedthe next time is made different from the last time in the correlationoperation for obtaining the phase difference. In a case where it isacceptable for the focus detection precision to be low, the number ofsteps is increased; in a case where a high precision is required forfocus detection, the number of steps is reduced. This makes it possibleto perform calculation suited to the required focus detection precisionand to perform calculations more quickly.

[0181] Further, in accordance with this embodiment, the size of the areaof a plurality of image signals used in the correlation operation thenext time is made different from the last time in the correlationoperation for obtaining the phase difference. As a result, it ispossible to change the area of the subject used in sensing focus and theprecision with which focus is sensed is raised. In addition, needlesscalculations can be eliminate and, hence, calculations can be performedat higher speed.

[0182] Further, in accordance with this embodiment, whether or not ashift operation in a direction substantially orthogonal to the directionof the phase shift produced in dependence upon the defocusing quantityof the optical system is selected in the correlation operation forobtaining the phase difference. As a result, by performing a shiftoperation in a direction orthogonal to the direction of the phase shiftproduced in dependence upon the defocusing quantity, it is possible toforego the shift operation in this direction in a case where focusdetection precision declines. The focus detection precision is raised asa result. In addition, since needless calculation can be eliminated,calculation time can be shortened.

[0183] Further, in accordance with this embodiment, the correlationoperation is performed upon downsampling the data of a plurality ofimage signals in an area used in the correlation operation. By selectingwhether or not to perform downsampling based upon informationrepresenting the preceding detection of focus, the downsampling methodused in the correlation operation the next time can be different fromthe last time in dependence upon the focus detection precision required.This makes it possible to eliminate needless calculation the next timefocus is detected and, hence, to shorten calculation time.

[0184] Further, in accordance with this embodiment, owing to the factthat the information representing sensing of focus is the result ofsensing focus, the state of defocusing can be determined from thepreceding result of focus detection. As a result, the method ofdownsampling data used in the correlation operation can be changedaccordingly, it possible to eliminate needless calculation and, hence,to shorten calculation time.

[0185] Further, in accordance with this embodiment, owing to the factthat the information representing sensing of focus is the number oftimes focus is sensed, the method of downsampling data used in thecorrelation operation can be changed in dependence upon the number oftimes focus is sensed. In case of the initial sensing of focus when theprobability of a large amount of defocusing is high, downsampling isperformed. Downsampling is eliminated as the number of times sensing isperformed increases. This makes it possible to eliminate needlesscalculation and, hence, to shorten calculation time.

[0186] [Switching Correction Equation]

[0187] A case in which the S/N ratio of the output signal from the imagesensing element (CCD 9 in FIG. 1) is poor will be described.

[0188] When the output signal of the image sensing element has a poorS/N ratio, a variance in the plurality of phase differences δ1, δ2, δ3,δ4 used in finding the phase difference δ after a correction increases.If the variance is large in comparison with a phase-shift component dueto relative movement between the camera and subject, using thecorrection calculation equation (2) to obtain the phase difference δafter correction is inappropriate since only the weighting of δ2 and δ3will be large.

[0189] Accordingly, a first example of switching of the correctioncalculation equation is such that if the S/N ratio exceeds apredetermined value, the correction calculation equation (2) is madeEquation (3) below, in which the arithmetic mean of the plurality ofphase differences δ1, δ2, δ3, δ4 is adopted as the phase difference δafter correction.

δ=(δ1+δ2+δ3+δ4)/4  (3)

[0190] The results of automatic exposure for auto focus are used asmeans for judging the S/N ratio of the output signal from the imagesensing element.

[0191] Automatic exposure for auto focus is performed using the signalof an area that same as that used in sensing of focus before the leftimage 1 of the phase-difference detecting signal is captured, namely inan optical state identical with that when the focal-point sensingdiaphragm 3 is inserted into the optical path of the taking lens, theright pupil 3 b of the lens is blocked by the light shield 5, theoptical image comprising the ray bundle that has passed through the leftpupil 3 a of the lens is formed on the CCD and the phase-differencedetecting signal is captured. The output signal of the CCD is subjectedto processing similar to the processing for obtaining thephase-difference detecting signal. If the exposure level found from thissignal is lower than a predetermined value, the exposure conditions arechanged.

[0192] In pupil time-sharing phase-differential auto focus, thefocal-point sensing diaphragm 3 is inserted into the optical path of thetaking lens. During capture of the phase-difference detecting signal,therefore, the F-number is fixed and cannot be changed. This means thatadjustment of exposure is performed by adjusting storage time in the CCDand output gain of the CCD. If the adjustment time is longer than apredetermined time, however, there will be cases where there adeleterious effect upon sensing of focus even phase-difference detectionerror to due relative movement between the camera and subject iscorrected by Equation (2). Consequently, there are instances where theadjustment of exposure is carried out by adjusting the output gain ofthe CCD.

[0193] Thus, in a case where the exposure adjustment is performed by theoutput gain of the CCD, the S/N ratio of the image is degraded when thegain is raised. Accordingly, in a case where the result of automaticexposure for auto focus is darker than a predetermined value incomparison with proper exposure, the image obtained will have a poor S/Nratio. In such cases, therefore, a decision is made to the effect thatthe image will be one having a poor S/N ratio.

[0194] More specifically, in a case where the exposure of thephase-difference detecting signal is less than proper exposure, theimage obtained by the CCD will be dark and will exhibit a poor S/Nratio. Accordingly, in a case where the phase-difference detectingsignal is captured under such dark conditions, it is judged that the S/Nratio is poor and the correction calculation equation is switched fromEquation (2) to Equation (3).

[0195] The amount of lens feed necessary for achieving the in-focusstate is found from the thus obtained phase difference δ, which has beencorrected for relative movement between the camera and subject, focuscontrol of the optical system is performed accordingly and the in-focusstate is achieved by repeating the sensing of focus until the amount ofdefocusing falls below a predetermined allowed amount. The picture isthen taken.

[0196] In the case of a camera system in which the amount of defocusingfalls below a predetermined allowed amount by the first focus sensingoperation and control of optical system focus, the picture may be takenwithout performing sensing of focus a second time for the purposes ofverification.

[0197] After the in-focus state is thus achieved, the focal-pointsensing diaphragm 3 is withdrawn from the optical path of the takinglens and then the picture is taken. In order to record the image, thedata is read out of the CCD, subjected to image signal processing by thedigital signal processor 12 and, if necessary, subjected to processingsuch as image data compression. The resulting image is recorded on arecording medium via the card slot 13.

[0198] This image data is subjected to video processing by the digitalsignal processor 12 in order to be displayed in the viewfinder. Theprocessed data is displayed in viewfinder 18 via the VRAM 21. Thephotographer can thus check the image of the subject.

[0199] Though a method of using the results of automatic exposure forauto focus has been described in this embodiment as a method ofevaluating the S/N ratio of the output signal from the image sensingelement, the S/N ratio may be evaluated using the last results ofsensing focus. In a case where the variance of a plurality of previousphase differences δ1, δ2, δ3, δ4 exceeds a predetermined value, it isjudged that the S/N ratio is poor and the correction calculationequation is changed from Equation (2) to Equation (3).

[0200] Further, the S/N ratio may be evaluated based upon a change inthe corrected phase difference δ, which is the result of sensing focusthe preceding time. If the result of sensing focus does not converge tofall within the allowable amount of defocusing regardless of repeatingthe sensing of focus a prescribed number of times or more, there areoccasions where the cause is a poor S/N ratio. In such cases thecorrection calculation equation is changed from Equation (2) to Equation(3).

[0201] A second example of switching the correction calculation equationaccording to this embodiment will now be described.

[0202] This second example of switching the correction calculationequation in such that when the output signal of the image sensingelement exhibits a poor S/N ratio, the number of times thephase-difference detecting signal used to obtain the phase difference δis captured is increased.

[0203] Only the differences between this method of switching thecorrection calculation equation and the first example of the switchingmethod will be described.

[0204] When the output signal of the image sensing element has a poorS/N ratio, a variance in the plurality of phase differences δ1, δ2, δ3,δ4 used in performing the correction calculation increases. Theprecision with which focus is sensed declines as a result. Accordingly,if the number of captures of the phase-difference detecting signalnecessary for obtaining a plurality of phase differences is increased,the number of phase differences is increased and the average valuethereof is found as the corrected phase difference δ, then it possibleto suppress a decline in focus sensing precision due to a largevariance.

[0205] Accordingly, when S/N ratio is judged to be poor from the resultsof automatic exposure for auto focus or from the preceding results ofsensing focus, the number of times the phase-difference detecting signalis captured is increased to ten times from the usual five, images L1,R2, L3, R4, L5, R6, L7, R8, L9, R10 for which the pupil has beenswitched in a time series are captured and phase differences δ1-δ9between L1 and R2, R2 and L3, L3 and R4, R4 and L5, R5 and L6, L6 andR7, R7 and L8, R8 and L9, L9 and R10, respectively, are obtained. Thearithmetic mean of the plurality of phase differences is adopted as thecorrected phase difference δ, as indicated by correction calculationequation (4) below.

δ=(δ1+δ2+δ3+δ4+δ5+δ6+δ7+δ8+δ9)/9  (4)

[0206] Though a case in which the number of times capture is performedis increased from five to ten has been described, the number of capturesis not limited to this number.

[0207] Thus, in accordance with the switching of the correctioncalculation equation of this embodiment, as described above, thecalculation equation for obtaining the amount of defocusing from aplurality of phases is changed in dependence upon the output of theimage sensing element. For example, when the S/N ratio is poor, thearithmetic means of the plurality of phase differences is changed. As aresult, a decline in the precision with which focus is sensed can besuppressed.

[0208] Further, by increasing the number of times an image for obtainingthe phase difference is captured, it is possible to suppress the effectsof a variance in a plurality of phase differences caused by poor S/Nratio and a decline in the precision with which focus is sensed can bereduced.

[0209] It should be noted that the system comprising the functionalblocks in FIG. 1 may be implemented by hardware or by a microcomputersystem comprising a CPU and memory. When implementation is by amicrocomputer system, the memory constitutes a storage medium accordingto the present invention and a program for executing the above-describedprocessing is stored on this storage medium. A semiconductor memory suchas a RAM or ROM, an optical disk, a magneto-optic disk or a magneticmedium may be used as the storage medium and these may be used in theform of a CD-ROM, floppy disk, magnetic tape or non-volatile memorycard, etc.

[0210] The present invention is not limited to the above-embodiments andvarious changes and modifications can be made within the spirit andscope of the present invention. Therefore, to apprise the public of thescope of the present invention, the following claims are made.

[0211] In the embodiment described above, the left and right pupils areopened alternately in order to detect phase difference. However, thepresent invention is applicable also to an arrangement in which, ratherthan the left and right pupils being opened alternately, the respectiveray bundles from the left and right pupils are received simultaneouslyby image sensing elements at different positions in the imaging plane.

[0212] Further, phase detection in the present invention is applicablein schemes other than the autotely, the respective ray bundles from theleft and right pupils are received simultaneously by image sensingelements at different positions in the imaging plane.

[0213] Further, phase detection in the present invention is applicablein schemes other than the auto focus scheme.

[0214] Further, all or part of the claims or embodiments may form asingle apparatus, an entity to be combined with another apparatus, or anelement constituting an apparatus.

[0215] Further, the present invention is applicable to various types ofcameras such as a video movie camera, a video still camera, a camerathat uses a silver-halide film and has an image sensing element forcapturing a photographic image, a lens-shutter camera and a surveillancecamera; image sensing devices, optical devices and other devices besidescameras; devices applicable to these cameras, image sensing devices andother devices; and elements constituting these devices.

What is claimed is:
 1. An apparatus comprising: (A) an image sensingdevice for receiving light from an object, said image sensing devicehaving a plurality of pixels for receiving light from the object uponseparating the light into respective ones of different color components;(B) an optical guidance device for accepting light rays in differentdirections from the same part of the object and guiding the light raysto said image sensing device; and (C) a phase-difference determiningdevice for determining a phase difference in an output signal of saidimage sensing device with respect to each of the light rays accepted inthe different directions by said optical guidance device, saidphase-difference determining device determining the phase differencebased upon a signal obtained by combining output signals of pixels,among said plurality of pixels, that correspond to prescribed differentcolor components; wherein said image sensing device changes anoutputting manner of the output signal based on whether the lightreceived from the object is dark or not.
 2. The apparatus according toclaim 1, further comprising: a signal processor for processing an outputsignal of said image sensing device to an image signal for photography;wherein said phase-difference determining device determines the phasedifference based upon an output signal of said image sensing device notsubjected to the processing by said signal processor.
 3. The apparatusaccording to claim 1, further comprising a signal processor forprocessing an output signal of said image sensing device to an imagesignal for photography; wherein the combining is performed based uponoutput signals of the pixels not subjected to the processing by saidsignal processor.
 4. The apparatus according to claim 1, furthercomprising a focusing device for focusing an imaging optical systembased upon results of determination by said phase-difference determiningdevice.
 5. The apparatus according to claim 1, further comprising afocusing device for focusing an image sensing optical system based uponresults of determination by said phase-difference determining device. 6.The apparatus according to claim 1, wherein said phase-differencedetermining device performs addition as the combining of the outputsignals.
 7. The apparatus according to claim 1, wherein saidphase-difference determining device performs addition and averaging asthe combining of the output signals.
 8. The apparatus according to claim1, wherein said phase-difference determining device applies weighting tothe output signals of the pixels corresponding to the prescribed colorcomponents when the combining of the output signals is performed.
 9. Theapparatus according to claim 8, wherein said phase-differencedetermining device changes the method of applying weighting independence upon an illuminating light illuminating the object.
 10. Theapparatus according to claim 1, wherein said optical guidance deviceselectively performs the guidance, to said image sensing device, oflight rays accepted in different directions from the same part of theobject.
 11. The apparatus according to claim 1, wherein said opticalguidance device selectively and repeatedly performs the guidance, tosaid image sensing device, of light rays accepted in differentdirections from the same part of the object.
 12. The apparatus accordingto claim 1, wherein said optical guidance device includes a movableportion for selectively performing the guidance, to said image sensingdevice, of light rays accepted in different directions from the samepart of the object.
 13. The apparatus according to claim 1, wherein saidphase-difference determining device selects pixels used in determiningsaid phase difference from among said plurality of pixels.
 14. Theapparatus according to claim 1, wherein said phase-differencedetermining device selects pixels, which are used in determining thephase difference, from among said plurality of pixels in dependence uponphotographic conditions.
 15. The apparatus according to claim 1, whereinsaid phase-difference determining device employs a plurality ofcalculation methods used in the determination of phase difference. 16.The apparatus according to claim 1, wherein said apparatus is an opticalapparatus.
 17. The apparatus according to claim 1, wherein saidapparatus is a camera.
 18. An apparatus comprising: (A) an image sensingdevice for receiving light from an object; (B) a signal processor forprocessing an output signal of said image sensing device, which hasreceived the light from the object, to an image signal for photography;(C) an optical guidance device for accepting light rays in differentdirections from the same part of the object and guiding the light raysto said image sensing device; and (D) a phase-difference determiningdevice for determining a phase difference in an output signal of saidimage sensing device with respect to each of the light rays accepted inthe different directions by said optical guidance device, saidphase-difference determining device determining the phase differencebased upon an output signal of said image sensing device not subjectedto the processing by said signal processor, wherein said image sensingdevice changes an outputting manner of the output signal based onwhether the light received from the object is dark or not.
 19. Theapparatus according to claim 18, wherein said image sensing device has aplurality of pixels for receiving light from the object upon separatingthe light into respective ones of different color components.
 20. Theapparatus according to claim 18, further comprising a focusing devicefor focusing an imaging optical system based upon results ofdetermination by said phase-difference determining device.
 21. Theapparatus according to claim 18, further comprising a focusing devicefor focusing an image sensing optical system based upon results ofdetermination by said phase-difference determining device.
 22. Theapparatus according to claim 18, wherein said optical guidance deviceselectively performs the guidance, to said image sensing device, oflight rays accepted in different directions from the same part of theobject.
 23. The apparatus according to claim 18, wherein said opticalguidance device selectively and repeatedly performs the guidance, tosaid image sensing device, of light rays accepted in differentdirections from the same part of the object.
 24. The apparatus accordingto claim 18, wherein said optical guidance device includes a movableportion for selectively performing the guidance, to said image sensingdevice, of light rays accepted in different directions from the samepart of the object.
 25. The apparatus according to claim 18, whereinsaid image sensing device has a plurality of pixels for receiving lightfrom the object, and said phase-difference determining device selectspixels used in determining said phase difference from among saidplurality of pixels.
 26. The apparatus according to claim 18, whereinsaid image sensing device has a plurality of pixels for receiving lightfrom the object, and said phase-difference determining device selectspixels, which are used in determining the phase difference, from amongsaid plurality of pixels in dependence upon photographic conditions. 27.The apparatus according to claim 18, wherein said phase-differencedetermining device employs a plurality of calculation methods used inthe determination of phase difference.
 28. The apparatus according toclaim 18, wherein said apparatus is an optical apparatus.
 29. Theapparatus according to claim 18, wherein said apparatus is a camera.